Meta-analysis

We absolutely need the mechanics and the connections, but we also need the big picture (meta-analysis) - they are complementary

Synthesis is the combination of two or more entitites - studies, regions, etc., the integration of evidence. Maps are useful - where the work has been done.

Zen - mind and body, zen synthesis - mind is synthesis, body is data.

Systematic review are similar to meta-analysis. A meta-analysis always has statistics, not always a p-value but gets effect sizes, variance, etc. The systematic review is one-step below meta-analysis.

The goal is to identify an effect size or average effort that matters to the big picture.

Synthesis is a counterweight to hyper-specialization - reduces silo effect among disciplines and sub-disciplines. Diversity promotes discovery, conceptualizes complexity, and copes with data deluge.

Synthesis will shift your view of statistical significance.

Effect sizes much more important than p-values, the strength of the difference - important for primary studies.

Synthesis framing - from low to hi - natural concepts -> functional concepts -> integrative concepts.

Steps of meta-analysis - questions, set of studies, predictions (summarize knowledge, meta-analysis). Each study is a sample point.

Systematic review has ‘pre-specified eligibility criteria’ aka your search terms for papers in the primary literature. This commonly uses a PRISMA report or approach, e.g., we started with 2000 papers, filtered down to 20. Can use Web of Science, SCOPUS, Google Scholar - use all. There are R packages to do this!!

Meta-analysis/systematic review also very important for gap analysis.

Meta-analysis always has an effect size, what was the strength of evidence plus variance and a measure of difference.

Effect size measured as:

Workflow for systematic review

Step 1:

Full workflow here

  1. pick an engine, document all search instances and record total number of returned hits
  2. explore search term sensitivity and synonyms in detail
  3. test >1 bibliometric resource
  4. do not add publications ‘by hand’ or haphazardly
  5. do check select publications (seminal and/or top cited) to calibrate search terms
  6. use abstract surveys to ensure you are capturing the correct set of literature
  7. do a search for key key terms by meta-analysis, review, and systematic review to avoid re-inventing the wheel or to ensure you are on the right track
  8. review boolean operators in searching

Step 1 notes: Search multiple search terms, don’t forget to remove duplicates.

In Web of Science - View Citation Report top right for summary of search results!!!! Export as text or csv.

The idea is to saves results from multiple searches (as text file), then cross-references for duplicates, relevance (by title/abstract), make final list (sort)

Step 2: sort - do a lot of this by hand, then you can make a flowchart diagram

step 2 PRISMA - preferred reporting items for systematic review and meta-analysis

library(tidyverse)
library(PRISMAstatement)

prisma(found = 156,
       found_other = 0,
       no_dupes = 53, 
       screened = 53, 
       screen_exclusions = 0, 
       full_text = 53,
       full_text_exclusions = 24, 
       qualitative = 0, 
       quantitative = 29,
       width = 800, height = 800)

Step 3: Synthesis

  1. extract evidence from the filtered list of eligible primary studies
  2. tabulate the evidence
  3. Data QA/QC and ensure you have what you need to do a meta.

Step 4: Summarize

You can use an evidence map to summarize. This is also where a map (or heatmap) or stacked barplot can go. Summarizes how well we have explored these topics.

Step 5: Statistics

All stats are a tool and a means to an end, not the end. In most cases, a simple test is appropriate (t-test, ANOVA, Chi-squared test) where each study is an independent sample.

Effect size plots can be viewed with a forest plot.

Reference: Meta-Analysis with R, look for e-book though.